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Introduction 


In 2010 Wilburn Elementary received a 5-year Teacher 
Incentive Fund (TIF) grant through the US Department of 
Education. The award was $1,809,369 over five years, or 
about $360,000 per year. Carryover of about $500,000 is 
being spent on TIF in the 2015-16 school year. This 
program supports efforts to develop, implement, and 
sustain performance-based teacher and principal 
compensation systems in high-need schools. Performance- 
based pay systems are designed to increase the number of 
effective teachers instructing disadvantaged students and 
thereby improve student achievement and closing 
achievement gaps by exposing students in those schools to 
more effective teaching and administrative support. 


In 2009-10, two-thirds of the student population at 
Wilburn was considered educationally disadvantaged. 
Additionally, Wilburn was one of the four lowest- 
performing elementary schools in WCPSS. At Wilburn, 
TIF funding was used to support the Teacher 
Advancement Program (TAP), freeing up Title I dollars 
that had been used previously. For 2014-15, Wilburn was 
in its seventh year of implementing TAP, and fifth year of 
TIF funding. 


The Teacher Advancement Program (TAP) is a 
comprehensive school reform initiative designed to attract, 
retain, and motivate quality teachers and teaching. TAP is 
one of the models accepted by TIF as a method to achieve 
its goals. 
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Abstract 


Wilburn implemented the Teacher 
Advancement Program (TAP) from 2008-09 
through 2014-15. TIF funding supported the 
performance pay component since 2010-11. 
All TAP principles were implemented. 
However, challenges occurred related to staff 
turnover, scheduling, and the extra work 
involved. Many teachers did not find the TAP 
model very helpful. The TAP rubric and 
model lessons (with coaching) were 
considered the most positive aspects. Relative 
to the comparison schools, the pattern of K-2 
literacy results was mixed for 2013-14 and 
somewhat stronger in 2014-15. Wilburn also 
showed similar patterns for the percentage of 
students scoring proficient on the EOG from 
2007-08 (before TAP) to 2011-12 and 2012- 
13 to 2014-15 (with the revised EOG). 
Wilburn’s growth results on EVAAS were at 
or above state expectations each year except 
for grade 4 in both subjects in 2015. Results 
were similar to comparison schools (with 
somewhat stronger results in 5th grade math). 
Overall, Wilburn achievement results have not 
been consistently stronger with TAP 
compared to similar schools without TAP. 
The cost/benefit of the model was not clearly 
favorable. 
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The four principles upon which TAP is based are: 


e multiple career paths, 

e ongoing, applied professional growth, 

e instructionally-focused accountability, and 

e performance-based compensation (Agam, Reifsneider, & Wardell, 2006). 


It is critical that all four principles of TAP are fully implemented to ensure success of a school or 
district. 


As part of TAP, Wilburn teachers have had the opportunity to participate in multiple career paths 
(as Master or Mentor teachers), receive financial rewards based upon their evaluations and 
student growth, receive job-imbedded professional development that is relevant, and receive 
evaluations based on an instructionally focused rubric. Mentor teachers take on additional 
responsibilities, including leadership responsibilities and mentoring and observing other 
teachers, and thus receive supplemental pay. Master teachers, on the other hand, are non- 
classroom based teachers who drive the professional development for the school as well as 
conduct teacher observations. Master and Mentor teachers at Wilburn were chosen through a 
competitive, performance-based selection. 


TAP was introduced to Wilburn Elementary School by staff vote in the 2008-09 school year to 
improve the effectiveness of Wilburn's teaching staff, to offer additional support in implementing 
other initiatives such as the Sheltered Instruction Observation Protocol (SIOP®), and to decrease 
staff-development related absences, when possible. Unlike most schools adopting TAP, Wilburn 
decided not to take a full year for planning (including hiring, training, and certifying the 
leadership team). Instead, they combined their planning year with their first year of 
implementation. For the first year of TAP implementation (2008-09), mathematics was the focus 
improvement area. For all subsequent years, the focus was reading, given that Wilburn students’ 
proficiency in reading remained very low. Improving students’ reading abilities was also seen as 
one way to help students increase their mathematics proficiency levels. In Years 5, 6, and 7 
(2012-15), there was an additional focus on improving students' math knowledge and skills. 


In November 2010, due to continuing low student performance, Wilburn was designated as one 
of four "Renaissance Schools" by the Wake County Public School System (WCPSS) as part of 
the district’s Race to the Top initiative’. The Renaissance model included signing bonuses, 
performance-based compensation, an infusion of technology, and an extra teacher per grade level 
to reduce class sizes. All staff were required to reapply for their jobs. Wilburn stayed with the 
TAP model and therefore differed somewhat from the model used in the other three Renaissance 
schools (Townsend, 2013). Renaissance funds supplemented TIF funds in providing bonuses to 
staff at Wilburn. 


‘ The other Renaissance Schools were Barwell Road Elementary, Brentwood Elementary, and Creech Road 
Elementary. (See Townsend, Baenen, and Germuth, 2015.) 
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Evaluation Methods 


The evaluation of TAP/TIF at Wilburn focused on both program implementation and student 
outcomes across the five years of the TIF grant. The evaluation questions that guided the study 
and the success targets set by Wilburn initially are shown in Appendix A. Both qualitative and 
quantitative data were utilized to evaluate the program. More specifically, the data sources used 
to compile this report were garnered from: 


e The program's grant application, 

e Meetings with the TIF Grant Coordinator at WCPSS and the Senior Director of Program 
Accountability in the Data, Research, and Accountability department at WCPSS; 

e Focus groups with teachers and the three Master teachers; 

e Observations of cluster meetings, Leadership Team meetings, and classroom teaching; 

e Teacher surveys 2011-14. 

e Document review, including a review of the TAP manual, a description of the program 
components, the TAP lesson rubric, and the TAP teacher evaluation rubric; 

e North Carolina Mathematics and Reading End-of-Grade Test data for 2007-08 through 
2014-15 (grades 3-5); Dynamic Indicators of Basic Early Literacy Skills (DIBELS) 
assessment data for 2013-15 (grades K-2); and 

e Measures of Academic Progress (MAP) data for 2013-15 (grades K-5). 


Since Wilburn was the only WCPSS school involved in TAP/TIF, it was not feasible to run 
statistical tests about the likelihood differences were significantly different than would be 
expected. A set of similar schools (the other three schools involved in Renaissance) were used 
for comparison to ground our conclusions about patterns of change over time since the main 
difference was TAP/TIF rather than just Renaissance as an improvement model. Thus, this was a 
descriptive study. 


Table 1 
Nature of the Data Provided and Valid Uses 


Research Design Conclusions that Can be Drawn 


QO Experimental We can conclude that the program or policy caused changes in 
outcomes because the research design used random assignment. 
O Quasi-Experimental We can reasonably conclude that the program or policy caused 
changes in outcomes because an appropriate comparison strategy 
was used. 
VI Descriptive These designs provide outcome data for the program or policy, but 
MI Quantitative differences cannot be attributed directly to it due to lack of a 
M Qualitative comparative control group. 


Sources: List, Sadoff, & Wagner (2011) and What Works Clearinghouse (2014). 


EVAAS ratings used regression analyses to determine whether growth at each school met state 
standards. In addition, TAP schools have their own EVAAS formulas based on their standards 
for TAP schools nationally. 


Therefore, results reflect the patterns seen at Wilburn over time, but they are not generalizable to 
how the model would work in another WCPSS school or elsewhere. Other factors at the school 
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(such as the addition of the Renaissance effort and the staff turnover it caused), as well as the 
initial quality of the teachers, could impact results in addition to TAP and TIF. 


MGrate)Caritaatectatiyn 


All of the principles of TAP were implemented each year, with multiple career paths; ongoing, 
applied professional growth; instructionally-focused accountability; and performance-based 
compensation utilized. The main challenges affecting implementation of TAP were the lack of 
experience for Master and Mentor teachers, the inability to conduct evaluations and post-evaluation 
feedback sessions in a timely manner, staff turnover at all levels, and the extra requirements of TAP. 


Teachers’ average ratings on survey items were neutral to slightly positive about TAP. The 
percentage of items with positive average ratings increased between spring 2012 and spring 2014, 


especially for items related to supporting students who needed extra help and the evaluation process. 
A majority of teachers in focus groups, especially those new to teaching, found the TAP Instructional 
Rubric quite helpful in that it broke down lesson planning into core components. Despite this, many 
found TAP a burden overall and questioned the validity of the TAP evaluation instrument ratings. 
None of the teachers interviewed credited TAP with improving their content knowledge and only 
about half believed that it had improved their practice/pedagogy. 


Professional Advancement 


Interviews and observations support that teachers were able to advance professionally without 
having to leave the classroom. Based on interest, ability, and accomplishments, all teachers were 
given the opportunity to advance professionally by becoming Mentor teachers. Becoming a 
Master teacher was also a possibility. Given this new career path, Mentor teachers did not have 
to leave the classroom but could continue to work with students while also having more of an 
opportunity to work with their colleagues. 


Master and Mentor teachers designed and delivered professional development to cluster groups 
and individual teachers while determining content knowledge and pedagogical skills that should 
be shared/enhanced as part of cluster meetings. All were part of the school’s leadership team 
and, as such, participated in setting school goals, supporting professional development, and using 
data to assess how students were progressing and what strategies were needed to improve student 
achievement. 


Ongoing, Applied Professional Growth 


During 2014-15, teachers advanced professionally without having to leave the school, spending 
90 minutes per week in professional learning as part of cluster team meetings. These meetings 
were held within each grade-level team before the start of the school day all years except 2012- 
13, when they were held during the school day. 
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Cluster meetings focused on needs based on specific analyses of End-of-Grade data, Northwest 
Evaluation Association's Measures of Academic Progress (MAP) data, and formative classroom 
assessments. Teachers presented student data at weekly cluster meetings and discussed findings 
based on these data. Teachers also collected data from their assessment of students on learning 
skills emphasized during cluster meetings and then shared these data as well. 


Master and Mentor teachers oversaw TAP activities and provided leadership and support through 
group and individual coaching. Ongoing classroom-based support was provided through team 
teaching and classroom demonstrations. Master and Mentor teachers met weekly to determine 
which strategies and skills to emphasize in order to increase students' reading and math 
achievement. Strategies were field-tested with Wilburn students at different grade levels before 
they were shared with other teachers. In addition, Master and Mentor teachers offered team 
teaching, classroom demonstrations, and feedback as part of pre- and post-conferences, to help 
improve their colleagues’ teaching. 


Instructionally-Focused, Performance-Based Accountability 


Teachers were held accountable for meeting the TAP Teaching Skills, Knowledge, and 
Responsibilities Performance Standards and were evaluated based on those standards using the 
TAP Evaluation Rubric. 


Each teacher was evaluated six times each year of TAP, with two announced observations and 
four unannounced observations. Either the TAP evaluation rubric or the teacher evaluation rubric 
developed for educators across North Carolina was utilized for observations. All administrators, 
Master, and Mentor teachers were trained on the use of the TAP evaluation rubric before they 
could use this instrument when conducting observations. Pre-conferences were provided prior to 
the two announced evaluations. All evaluations ended with a post-conference where teachers 
were provided detailed information about their lesson and how it was rated. In 2014-15, about 
half of the post-conferences were conducted within 72 hours of observations. In 2013-14, almost 
all were conducted within 72 hours. In the years prior, very few were conducted within 72 hours 
and some were conducted as late as three months after the observation. 


Under TAP, teachers were also evaluated based on the academic growth of ee 
their students and whole student growth. All teachers were eligible for 


performance-based compensation based on these criteria. Performance- lal a 


based compensation was provided each year of TAP and TIF. Salary six times each 
augmentation was given to Master and Mentor teachers in all years year of TAP, with 
according to their additional responsibilities and authority. two announced 
Classroom observations conducted by the evaluator suggested most observations and 
teachers were incorporating strategies promoted by TAP and providing four 


quality instruction to their students. Strategies were evident in observations 
of direct instruction and student work. In addition, classroom observations 
and student interviews revealed that teachers were deliberately focusing on 
the various aspects of the TAP Instructional Rubric to improve student ——_ 
engagement and understanding. 


unannounced 
observations. 
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Teacher Perceptions of TAP 
Teacher Surveys 


Throughout the TIF project, all teachers were surveyed periodically about their TAP experiences. 
As is shown in Table 1, teachers were first surveyed in May 2012, and then again in December 
2012, May 2013, and May 2014. Response rates were high for the spring and fall 2012 surveys 
(98% and 93%, respectively), slightly lower for the spring 2013 survey (84%), and again high 
(98%) for the spring 2014 survey. The survey was not completed in spring 2015 at the 
principal’s request because of other time commitments. 


Table 1 
Teacher Survey Administrations and Response Rates 


Year Fall Response Rate Spring Response Rate 
2011-12 --- May 2012 (98%, n = 44/45) 
2012-13 Dec. 2012 (93%, n=42/45) May 2013 (84%, n= 38/45) 
2013-14 --- May 2014 (98%, n=44/45) 
2014-15 --- --- 


Tables 2-4 show mean survey ratings from spring 2012 to spring 2013 (1-year change) and 
spring 2012 to spring 2014 (2-year change). Teachers were asked to indicate their agreement 
using a 5-point scale (1 = Strongly Disagree to 5 = Strongly Agree with 3= Neutral). Most 
ratings were in the neutral range (2.5 to 3.5). A series of t-tests were run to assess whether 
differences in ratings were significant. 


For the items related to TAP’s appropriateness and impact (see Table 2), 3 of 14 items increased 
significantly between 2012 and 2014 (see green and bold font in table), and none decreased 
significantly. All three items with significant increases had to do with knowing student needs and 
how to address them. The item with the biggest positive gain between 2012 and 2014 was "TAP 
has allowed me to better identify students needing extra support." All three items had the lowest 
ratings in spring of 2012; two of the three were no longer among the lowest in 2014. 


Another interesting finding is that the least change was seen for the two items related to whether 
TAP made respondents more effective as teachers and whether TAP would affect student 
achievement; both were relatively high in both 2012 and 2014. The highest average response 
from teachers in spring of 2014 addressed whether TAP was aligned with Common Core 
standards. 
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Table 2 
Mean Ratings from the TIF Teacher Survey: TAP Appropriateness and Impact 2012-2014 


Item Stem: “TAP professional development provided _— Spring Spring Spring Diff. Diff. 

by Master teachers:” 2012 2013 2014 2013-12 2014-12 

a. is aligned to our school's needs. 3.50 3.45 3.84 -0.05 0.34 

b. is aligned to Common Core standards. NA 3.68 3.91 NA 0.23 

c. helps me better engage my students. 3.23 3.29 3.49 0.06 0.26 

d. has provided me with techniques to help me 3.18 3.97 351 071 0.33 
better group students. 

e. has made me better able to motivate my 5.95 3.00 3.19 0.05 0.23 
students. 

f. has improved my teaching skills. 3.45 3.22 3.67 -0.24 0.22 

g. has made mea more effective teacher. 3.45 3.32 3.56 -0.14 0.10 

h. has allowed me to better identify students 5 68 7 8A 3.56 0.16 0.88 
needing extra support. 

i. has allowed me to better identify the type of 7.82 2 8A 3.50 0.02 0.68 
support my students need. 

j. is meeting my needs as a teacher. 2.95 2.74 3.30 -0.22 0.35 

k. has made me more knowledgeable about my > 86 3.95 3.33 0.08 0.47 
students. 

I. has improved my ability to help my students to 3.28 3.16 3.56 0 0.28 
use higher order thinking skills. 

m. has improved my ability to help my students to 3.14 ai 3.47 0.03 0.33 
use better problem-solving skills. 

n. wll help student achievement increase at 3.39 371 3.49 6.48 0.10 
Wilburn. 


Note: NA means not applicable; Item b was not included in the 2012 survey. A green and bold font means that 
differences were significant in the positive direction. Ratings of 3.5 or greater are shaded in gray. A5-point scale 
was used (1 = Strongly Disagree to 5 = Strongly Agree with 3 = Neutral). 

Source: TIF Teacher Survey 


In Table 3, the greatest positive changes related to the support and experience of Master and 
Mentor teachers and their ability to help teachers. The only negative change that was statistically 
significant related to 2012 and 2013 responses was item i ("The administration at my school is 
supportive of TAP."). However, the 2014 rating increased considerably over 2013 to a mean of 
4.21, which was consistent with other data indicating a more positive regard for the new 
administration's support of TAP. The highest rated item varied each year. In 2014, use of data to 
track student progress had the highest average response (4.31). 
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Teacher Ratings from the TIF Teacher Survey: TAP Knowledge, Use, and Support 2012-14 


nen 2012 2013 2014 


Spring Spring Spring Diff. Diff. 


| am highly knowledgeable of the TAP 


instructional rubric. 222 a8 gies 
| regularly use data to track my students 3.98 ADA 431 
progress. 

Master teachers are available to support me if | 3.36 3.92 A24 
need help. 

Mentor teachers are available to support me if | 3.27 3.50 4.00 
need help. 

Master teachers have the experience necessary 3.23 311 3.52 
to help me improve my teaching skills. 

Mentor teachers have the experience necessary 544 3.34 3.79 


to help me improve my teaching skills. 


The professional development designed and 
provided by Master teachers helps me improve 2.98 2.95 3.43 
my instruction. 


The professional development designed and 
provided by Mentor teachers helps me improve 2.80 3.16 3.38 
my instruction. 


The administration at my school is supportive of 


TAP. 4.02 3.53 4.21 


TAP is a positive aspect of being at Wilburn. 2.91 2.53 3.14 


2013-12 2014-12 
0.40 0.14 
0.26 0.33 
-0.44 0.87 
0.23 0.73 
-0.12 0.30 
0.21 0.65 
-0.03 0.45 
0.36 0.59 
-0.50 0.19 
-0.38 0.23 


Note: T-tests compared differences between 2012 to 2013 and 2012 to 2014. Green bold indicates significant 
differences in the positive direction. Red bold means that differences were significant in the negative direction. 
Ratings of 3.5 or greater are shadedin gray. 5-point rating scale: 1 = Strongly Disagree to 5 = Strongly Agree, with 


3 = neutral. 
Source: TIF Teacher Survey 


In 2012, the lowest mean ratings were for items related to the fairness of 
evaluations based on the TAP rubric, whether the teachers agreed with 
their ratings, and whether they had a say in the evaluation process (b, g, 
and j in Table 4). Significant increases were seen for these and other 
items related to teacher evaluations based on the TAP rubric in either 
2012-13 or 2013-14. All Master teachers were new in 2011-12, which 
meant they had to be trained and were new at using the observation 
rubric. Focus groups revealed that teachers were very unhappy with how 
observations were conducted in 2011-12 (see next section on Teacher and 
Principal Performance). 


Across all survey 
administrations, the 
percentage of items 
with positive mean 
ratings increased over 
time. 
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Table 4 
Teacher Ratings from the TIF Teacher Survey: TAP Observations 2012-14* 


Spring Spring Spring Diff. Diff. 


rene 2012 2013 2014 | 2013-12 2014-12 


a. | feel comfortable being evaluated using the 


TAP rubric as the basis for the evaluation. ee ze 327 a2 om 
b. Evaluations using the TAP rubric are being 559 448 349 0.66 ies 
conducted in a fair manner. 
c. Post-evaluation conferences are being 3.23 3.81 aa wae aay 


conducted in a timely manner. 


d. |believe that the Master teachers have the 
necessary training to conduct evaluations 3.09 3.29 3.65 0.20 0.56 
using the TAP rubric. 


e. I|believe that the Administrators at Wilburn 
have the necessary training to conduct 3.41 3.19 3.86 -0.22 0.45 
evaluations using the TAP rubric. 


f. | believe that the Mentor teachers have the 
necessary training to conduct evaluations 3.11 3.37 3.67 0.25 0.56 
using the TAP rubric. 


g. |agree with the TAP ratings | have received 


: 2.95 3.53 3.49 0.57 0.53 
for my evaluations. 


h. Iwas able to adequately learn the TAP 
instructional rubric before TAP evaluations 3.42 3.68 3.91 0.26 0.49 
were conducted. 


i. TAP evaluation feedback is meaningful to 


3.36 3.47 3.65 0.11 0.29 
me as a teacher. 
j. | feel like | have a say in the TAP evaluation 255 50 371 0.05 0.66 
process. 
k. The TAP evaluation process has helped me 3.05 3.13 3.62 0.09 0.57 


grow professionally. 
Note: Green bold means that differences were significant in the positive direction. Ratings of 3.5 or greater are 
shaded in gray. Teachers indicated their agreement using a 5-point scale (1 = Strongly Disagree to 5 = Strongly 
Agree with 3 = neutral). 
Source: TIF Teacher Survey 


The percentage of items with positive responses about TAP (at or above 3.5) increased 
dramatically between spring of 2012 and 2014, as is shown in Figure 1. A change in principals 
also occurred which could have had a positive impact in responses in 2013 and 2014. 
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Increase in Percentage of Items with Positive Responses Over Time 


70% 
60% 
50% 
40% 
30% 
20% 
10% 

0% 


25.7% 


11.4% 


Spring 2012 Spring 2013 


62.9% 


Spring 2014 


Interpretation Example: The percentage of the 35 items with positive mean responses (3.5 or greater on 
a 5-point scale) increased dramatically between spring of 2012 and spring of 2014 (from 11.4% to 62.9%). 


Focus Groups 


Focus groups of teachers were conducted each year at Wilburn. Some common concerns were 
that TAP was too much work, was stressful, that evaluations were not fair, and that post- 
conferences (where required) were not conducted in a timely manner. 


The majority of teachers interviewed indicated TAP had not been a 
positive experience for teachers at Wilburn, although some aspects 
of it (e.g., the TAP rubric and modeling of lessons) were useful. 
TAP was viewed more positively among beginning teachers than 
veteran teachers. Over half of the teachers participating in the focus 
groups expressed frustration with how TAP was implemented, 
indicating that TAP resulted in extra work on their part with very 
little perceived benefit to students. An example was having to use 
strategies with their students even when they believed the strategies 
did not fit what they were teaching. Many of these same teachers 
expressed the view that the turnover among the administration and 
Master teachers over the seven years during which TAP has been 
implemented had resulted in a lack of cohesion across the years 
and a fragmented approach to TAP. 
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Overall, the majority of 
teachers did not see TAP 
as a positive experience. 
Concerns focused on the 

extra work involved, 
timing and fairness of 
evaluations, and teachers 
having to apply strategies 
that they were taught, 
even when they did not 
seem appropriate. 
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Thus, most participants were negative about TAP and at least five 


teachers reported that colleagues who had left told them it was Most teachers felt the 
because of the stresses of TAP. While Wilburn teachers had seen TAP rubric and modeling 
the positives of having job-embedded professional development, and coaching were the 
some questioned whether money should be spent on the additional most valuable aspects of 
leadership positions supported by TAP (e.g., Master teachers) and TAP. Beginning teachers 
wanted teachers in such positions, including coaches, housed in and those new to 
actual classrooms and required to be more engaged in what is teaching elementary 
happening across the classrooms that they oversee. school were said to 
benefit the most. 


The three Master Teachers were also interviewed as a group in 
January 2015. They believed that TAP was especially valuable to 
beginning teachers or teachers new to the elementary school level. They also believed that all 
teachers benefitted by learning the TAP rubric as it gave all teachers at Wilburn a view of what 
constitutes good teaching and held them to the same standard. Whereas the Master teachers did 
realize that TAP required extra work on teachers’ parts, they believed that other issues beyond 
TAP have also led to teacher turnover. When asked what they would keep, the Master Teachers 
reported that they viewed the TAP rubric and coaching they have provided as the most useful 
aspects of TAP to teachers. They further suggested that, rather than a pay-for-performance 
model, Wilburn should adopt a sign-on/retention bonus for teachers who agree to work at or stay 
working at Wilburn. 


When examined in the light of other data, teacher concerns were supported. 


e In 2011-12, all of the Master teachers were new and TAP training was required before 
they could conduct any observations or give feedback. This meant the six observations 
and feedback sessions per teacher got a very late start. In addition, feedback sessions 
often took place a few weeks or even months after the observations. Continuity in Master 
teachers improved somewhat after 2011-12, but some turnover still occurred. Scheduling 
of observations and feedback sessions improved after 2011-12, with most post- 
conferences completed within 72 hours. However, some were still late. 


e One source of-extra work involved in TAP was the administration of the MAP 
assessments to assess student needs and progress in addition to the K-5 assessments 
(which continued to be required). Second, TAP added more training within the school, 
but some district training was still required. Third, TAP required six observations of each 
teacher with their instrument; the district used a different tool and observation schedule 
which also had to be met. Finally, sometimes applying the focus strategies from the 
training meetings required extra work—especially if they did not fit the lessons. 


In summary, TAP was implemented each year, but a lack of continuity in staffing and extra AP 
requirements made it difficult to consistently implement in a high-quality way. 
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Teacher and Principal Performance 


Since 2010-11, most teachers received a bonus at Wilburn based on meeting one or more of the 
following criteria: 
The percentage of teachers earning a rating of 2.5 or better on the TAP Instructional Rubric 
ranged from 90% to 100%. 
Among 4th and 5" grade teachers, 75-100% made the required classroom level growth each year; 
the percentage decreased somewhat over time at the fourth grade level. 
Wilburn met Whole School Growth EVAAS standards in three of the five years, earning all staff a 
bonus. 


Master teachers and principals were less likely than teachers to receive a bonus. 


While TAP and TIF hoped to lower teacher turnover, teacher turnover rates among those earning 2.5 
or better on the TAP Evaluation Rubric have been 25% or more in three of the five years. In 2012-13, 
the turnover rate was 3%, meeting the target of 5%. In 2014-15 the rate was 7%, just over the target. 


All Renaissance Schools had a bonus plan in 2014-15, but the TAP schools’ bonus plan differed 
from the other three Renaissance schools. Renaissance funds supplemented areas not funded by 
TAP. At Wilburn, all teachers earned bonuses based on two factors—the quality of classroom 
instruction (based on ratings of 2.5 or greater on the TAP Instructional Rubric), and whole 
school growth based on Grade 4 and 5 EVAAS scores. In grades 4 and 5, student growth at the 
classroom level on EVAAS was also a factor. Thus: 


e In grades K - 3 and for art, music, physical education, and special education teachers, the 
quality of classroom instruction and whole school growth each represented 50% of a 
teacher’s evaluation weight. 


e In grades 4 - 5, due to the availability of prior year End-of-Grade test scores, quality of 
classroom instruction was weighted 50%, student growth was weighted 30%, and whole 
school growth was weighted 20%. Whole school growth? was based on a value-added 
score of 3 or greater on an EVAAS scale with five categories rather than three (used 
across TAP schools). Student growth? for grade 4 and 5 teachers was based on an 
EVAAS value-added score of 3 or greater for their classroom. 


Quality of Instruction: For the 2014-15 school year, 49 of 53 (93%) teachers at Wilburn earned 
a rating of 2.5 or better on the TAP Instructional Rubric, qualifying them for at least a partial 


? Teacher performance is measured in North Carolina in part by using EVAAS scores at grades 4 and 5. A student's 
predicted score is based on his or her performance on previous tests. Student EVAAS scores are calculated as the 
difference between the actual test score and predicted score. Scores are then aggregated to the school level. A 
large negative mean indicates that students made less progress than expected; a large positive mean indicates that 
students made more progress than expected; and a mean of approximately 0.0 indicates that a group is 
progressing at an average rate. 

3 Using these EVAAS student data, teacher performance indices are then calculated by teacher and associated with 
meeting, or failing to meet, standards. 
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bonus. Since 2010-11 these percentages have ranged from 90% to 100%. For 2014-15, of the 
three Master teachers, only one earned the individual bonus. The principal and two assistant 
principals received bonuses as they met two of the three components, Whole School Growth 
(based on EVAAS scores) and Individual Growth (based on TAP Leadership Team Rubric 
scores). None met the requirements for School Growth based on their National Institute for 
Excellence in Teaching (NIET) Annual Review score. 


Table 5 
Percentage of Teachers Scoring 2.5 or Better on the TAP Instructional Rubric 


2010-11 2011-12 2012-13 2013-14 2014-15 
TIF Yr 1* TIF Yr2 TIF Yr 3 TIF Yr 4 TIF Yr5 
100% 97.5% 93% 90% 93% 


Source: School-level TAP Data. 
*Third year of TAP was first for TIF. 


Based on the criteria which applied to them, Master teachers were less likely to earn a bonus than 
regular classroom teachers. Of the three Master teachers at Wilburn each year, none to one 
received a bonus during the five years of TIF. 


Whole School Growth: From 2010-11 to 2014-15, Wilburn made Whole School Growth at the 
level required to earn teachers additional bonuses in 2011-12, 2012-13, and 2013-14, but not in 
2010-11 nor 2014-15. 


Student Growth for Individual Teachers: Among 4" and 5" grade teachers, 75-100% made 
the required classroom level growth each year to earn a performance award; the percentage 
decreased somewhat over time at the fourth grade level. The target was 100% for 2014-15. 


Table 6 
Percentage of 4°" and 5" Grade Teachers Making the Classroom Level Growth 
Required to Earn a Growth-Based Performance Award 


2010-11 2011-12 2012-13 2013-14 2014-15 
Grade Level 
TIF Yr 1* TIF Yr2 TIF Yr 3 TIF Yr 4 TIF Yr5 
Grade 4 NA* 100% 83% 75% 75% 
Grade 5 25% 100% 100% 100% 100% 


Source: EVAAS Data 2010-11 to 2014-15 
*Note: EVAAS data were not available for 4" grade teachers in Year 1 to make classroom level growth 
determinations 


Staff Turnover 
It was anticipated that TAP and TIF would lead to lower teacher turnover. Turnover was lowest 


in 2011-12 (3%) and 2014-15 (8%). The target of 5% was thus met for one of the five years. 
Turnover in principals, Master and Mentor teachers, and classroom teachers impacted the 
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continuity and implementation of TAP because of re-training needed for new staff as well as 
building trust and strong working relationships among colleagues. Other factors also contributed, 
such as stagnant pay rates for teachers. 


The highest turnover was seen when the Renaissance program was added, which required all 
current employees at each Renaissance school, including Wilburn, to reapply for positions for 
the 2011-12 academic year. Those not wishing to reapply could request a transfer to another 
school. This resulted in 67% of the staff being new at Wilburn for 2010-11, the second year of 
TIF. This disrupted the momentum of TAP becoming part of the school culture in that two thirds 
of the leadership and teaching corps were new and had to be trained in TAP. However, staff 
turnover of 25% or more was also evident in 2012-13 and 2013-14. 


Table 7 
Percentage of Teacher Turnover Among Teachers Earning A Score of 2.5 or Better 
on their TAP Evaluations 


2010-11 2011-12 2012-13 2013-14 2014-15 
TIF Yr 1* TIF Yr 2 TIF Yr 3 TIF Yr 4 TIF Yr 5 
# Teachers 33/49 1/31 20/66 16/63 4/52 
Turnover 67% 3% 30% 25% 8% 


Source: School-level TAP Data 

* 2011-12 was the first year of the Renaissance Schools Initiative, but staffing decisions began in 
spring of 2011. Therefore, the turnover rate from March of one year to the next is shown for 
2010-11 rather that 2011-12. 


Wilburn had three different principals since 2008-09 when the program began. The first principal 
was not re-hired with the beginning of Renaissance. The second principal was not very familiar 
with TAP. An interim principal was in place as they searched for a replacement (not shown). The 
third principal started in 2011-12 as the assistant principal and then was promoted to principal in 
2012-13. She continues to serve as principal at Wilburn. She brought stability to leadership at the 
school. 


Wilburn also had four assistant principals between 2008-09 and 2014-15. One became the 


principal (as mentioned above) and two served simultaneously, so the turnover was not as 
disruptive as for the principalship. 
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Figure 2 
Changes in Principals Across TAP and TIF 


2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15 


Principal 1 


Principal 2 
AP 1 
AP 2 
Principal 3 
AP 3 


AP 4 


For the 2014-15 school year, six Mentor teachers were hired (all from within the school) based 
on a process utilizing interview questions developed by TAP. These questions asked about 
designing, planning, and implementing instruction; classroom management; and working with 
teaching teams and parents. One Master teacher hired in 2012-13 school year remained as a 
Master teacher for the 2013-14 and 2014-15 school year. Two Master teachers hired in 2013-14, 
both of whom had served as teachers at Wilburn for many years and as Mentor teachers in the 
past, remained as Master Teachers for 2014-15. 


As is shown in Figure 3, 2014-15 was the first year that all Master Teachers had at least one 
years’ worth of experience in their role. Every other year, everyone was new to their role or one 
Master teacher carried over from the prior year. Mentor teachers also changed frequently year- 
to-year. 


Figure 3 
Changes in Master Teachers Across All Years of TIF 


2010-11 A B C 
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Note: Letters represent individuals who served as Master Teachers at Wilburn over time. Three served each year. 
Interpretation example. Master teacher C was the only one who served at Wilburn in the Master teacher role in 
both 2010-11 and 2011-12. 
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Student Achievement Results 


Compared to other Renaissance schools (the comparison group), K-2 literacy results were mixed in 
2013-14 and slightly stronger in 2014-15. 


Between 2007-08 (before TAP) and 2011-12, all of the Renaissance schools showed increases in 
proficiency in reading and math. Compared to other Renaissance schools, positive changes were 
smaller at Wilburn. From 2012-13 to 2014-15, small increases in proficiency occurred only at grade 
4 for both groups. Thus, TAP did not result in demonstrably greater improvements in reading 
proficiency over time. 


Overall, Wilburn growth results based on EVAAS were at or above state expectations each year 


except for grade 4 in both subjects in 2015. However, they were not consistently stronger with TAP 
compared to similar schools without TAP. The one exception was fifth grade math, where Wilburn did 
show a slight advantage compared to the other Renaissance schools over time. 


The percentage of students making targeted gains (five points) on fall to spring MAP assessments 
jumped greatly from 2011-12 to 2012-13 in both reading and math before dropping significantly in 
reading for 2013-14. The percentage of students meeting targeted gains in reading increased slightly 
in 2014-15 to 71%, short of the target of 80%. In math, for 2014-15, Wilburn fell just short of their 
math target of 80% of students making targeted gains in math (actual was 77%). 


MAP Results 


Measures of Academic Progress (MAP)* assessments are computer-adaptive interim assessments 
for use by schools and districts to measure individual students’ progress throughout the academic 
year in reading, language arts, and math. MAP assessments were given to K-5 students at 
Wilburn at set intervals to assess their learning progress in reading and math throughout the year. 
Results are provided as a RIT (Rasch Unit) scale, a stable equal-interval vertical scale that allows 
educators to compare students’ academic performance relative to national achievement and 
growth norms and specific state standards, including the Common Core State Standards (CCSS). 
Teachers used results to identify which students to target for additional support and in what 
areas. A project goal related to MAP was, by Year 5, for 80 or more of students to make a 5- 
point increase in their MAP reading scores from fall to spring testing and 80% or more to make a 
5-point increase in their MAP math scores from fall to spring testing. 


4 Measures of Academic Progress (MAP) was developed and is supported by Northwest Evaluation Associates. 
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As is shown in Table 8, the percentage of students making 5-point gains on fall to spring MAP 
assessments in reading showed a see-saw pattern of increases and decreases, jumping greatly 
from 2011-12 to 2012-13, dropping greatly for 2013-14, and increasing again in 2014-15 (to 
71%, just short of the target of 80%). In math, a large increase between 2011-12 and 2012-13 
was followed by small decreases in 2013-14 and 2014-15. Wilburn fell just short of their math 
target of 80% of students making 5-point gains in math (actual was 77%). 


Table 8 
Percent of Students Making 5-Point Gains on Fall to Spring MAP Assessments 


2010-11 2011-12 2012-13 2013-14 2014-15 
TIFYr1 © TIF Yr2 TIFYr3 © TIFYr4 TIF Yr5 
Fall MAP not 241/501 374/423. 313/580 ~=—- 287/405 


Reading given 48% 88% «54% 71% 
Math FallAPnot 318/501 390/445 418/497 242/316 
given 63% 88% 84% 77% 


Source: MAP Data. TAP was in place two years before TIF funding. MAP results were not available 
before 2010-11 


K-2 Results 


While not part of the stated goals for TAP, the Dynamic Indicators of Basic Early Literacy Skills 
(DIBELS)° were reviewed for context over the last two years. DIBELS is used to assess the 
literacy skills of WCPSS’ Kindergarten to Grade 2 students throughout the year. Higher scores 
are expected at each assessment point during the year. Increases in absolute terms were 
compared across schools since results represented the full population of the schools rather than a 
sample of students. 


In 2013-14, K-2 Composite Scores and Grade 2 DIBELS Oral Reading Fluency (DORF) scores 
reflected mixed results. Wilburn had higher gains than the other Renaissance schools in two of 
four comparisons (Kindergarten and Grade 1 Composite Scores) when compared to the other 
Renaissance Schools, but had lower gains for Grade 2 Composite and DORF Fluency (see Table 
9). 


Compared to WCPSS, increases were consistently lower than WCPSS, and therefore 
achievement gaps between Wilburn and WCPSS were not closed. 


5 DIBELS is supported by University of Oregon. For more information please see https://dibels.uoregon.edu/ 
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Table 9 
K-2 Literacy Skills 2013-14 


Fall to Spring 
Fall Winter Spring Increase 


Kindergarten 


Composite Score 


Wilburn 20.6 121.4 138.3 a ae 


Other Renaissance Schools 21.9 128.9 134.8 112.9 
WCPSS 35.0 147.7 154.2 119.2 
Grade 1 


Composite Score 


Wilburn 108.3 159.5 22 iii 


Other Renaissance Schools 115.0 159.6 168.9 53.9 

WCPSS 130.9 212.4 212.9 82.0 
Grade 2 

Composite Score 

Wilburn 162.9 197.0 226.1 63.2 

Other Renaissance Schools 158.0 196.2 221.7 63.7 

WCPSS 200.5 268.2 302.5 102.0 

DORF Fluency 

Wilburn 54.6 71.8 82.3 27.7 

Other Renaissance Schools 57.2 75.4 87.4 30.2 

WCPSS 75.7 95.8 110.1 34.4 


Source: DIBELS Next 2013-14 assessments. 
Note: Green shading indicates Wilburn increases fall to spring were larger than those of other Renaissance schools. 


In 2014-15, gains were slightly higher for Wilburn on four of five comparisons when compared 
to the other Renaissance Schools (Composite Scores at Kindergarten, and Composite and DORF 
Fluency results at Grades 1 and 2). In Kindergarten, students at Wilburn made gains that were 
markedly higher than those made by Kindergarten students at the other Renaissance schools and 
equivalent to the district average. The only case in which gains were smaller for Wilburn was 
DORF Fluency at grade 1, see Tables 10-12. Thus, results were stronger for Wilburn relative to 
the other Renaissance schools in 2014-15 than 2013-14 for DIBELS measures. 


Wilburn increases fall to spring were higher than those made by students in WCPSS as a whole 


in 3 of 5 comparisons; the other Renaissance schools had consistently lower increases than 
WCPSS overall at grades K-2. 
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Table 10 
Grade K Literacy Skills: DIBELS Next Composite 2014-15 


Fall to Spring 
Fall Winter Spring Increase 


Composite Score 

Wilburn 24.6 142.2 146.4 [TE 
Other Renaissance Schools 22.9 123.0 128.8 105.9 
WCPSS 36.2 152.0 157.2 121.0 


Source: DIBELS Next 2014-15 assessments. 
Note: Green shading indicates Wilburn increases fall to spring were larger than those of other 


Renaissance schools. 


Table 11 
Grade 1 Literacy Skills: DIBELS Next Composite and DORF 2014-15 


Fall to Spring 
Fall Winter Spring Increase 


Composite Score 


Wilburn 115.5 169.5 1679 [SZ 


Other Renaissance Schools 125.3 160.4 165.6 40.3 
WCPSS 139.2 219.2 215.1 75.9 
DORF Fluency 

Wilburn NA 37.8 55.0 17.2 
Other Renaissance Schools NA 35.2 53.8 18.6 
WCPSS NA 56.5 77.1 20.6 


Source: DIBELS Next 2014-15 assessments. DORF is not given until mid-year in grade 1. 
Note: Green shading indicates Wilburn increases fall to spring were larger than those of other Renaissance schools. 


Grade 2 students at Wilburn in 2014-15 gained over twice as many points on their Composite 
Score across the academic year compared to the other Renaissance Schools, and had greater 


gains than WCPSS second graders as a whole. 


Table 12 
Grade 2 Literacy Skills: DIBELS Next Composite and DORF 2014-15 


Fall to Spring 
Fall Winter Spring Increase 


Composite Score 


Wilburn 170.2 229.6 279.7 OE) 


Other Renaissance Schools 174.4 200.1 225.2 50.8 
WCPSS 212.2 271.0 304.3 92.1 


DORF Fluency 


Wilburn 58.6 79.5 95.9 Re ae 
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Other Renaissance Schools 59.2 74.6 83.8 24.6 


WCPSS 77.8 97.4 111.1 33.3 
Source: DIBELS Next 2014-15 assessments 
Note: Green shading indicates Wilburn increases fall to spring were larger than those of other Renaissance schools. 


In summary, Wilburn students showed mixed results compared to other Renaissance schools in 
2013-14. For 2014-15, K-2 results in literacy were generally stronger for Wilburn than the other 
Renaissance schools (in four of five comparisons). Achievement gaps closed somewhat between 
Wilburn and the district overall only in 2014-15 and in three of five comparisons made. 


End-of-Grade Results Grades 3-5 
Proficiency 


Comparing proficiency results within a grade over time gives us some sense of the impact of 
TAP on student proficiency. However, caution is necessary when attributing changes to 
increased teacher effectiveness in instruction given the challenge of high staff turnover in 2011- 
12 with the start of Renaissance. In addition, no comparisons can be made between 2011-12 and 
2012-13 because the curriculum, assessments, and standards for grade level achievement 
changed. Comparisons in gains in proficiency by grade were made over time from 2007-08 
through 2011-12 and from 2012-13 to 2014-15. No statistical tests were run since results 
reflected the full student population of each school and because of the other caveats listed above. 


Goals were set for mastery of EOG at the start of the program; goals were not changed with the 
adoption of a new curriculum and assessments. The reading target was 85% of students scoring 
at grade level or above and the math target was 90%. 


Reading 


Table 13 shows reading proficiency percentages at Wilburn and all Renaissance schools over 
time. The reading target in 2011-12 was 61% proficient; the target for 2014-15 was 85%. 
Neither target was met. The target was unrealistic given the change to the common core 
curriculum and assessment in 2012-13. In terms of improvements over time: 


e Comparisons by grade level over time (rows in the table) at Wilburn showed only one 
large increase in proficiency percentages in reading between 2007-08 and 2011-12 (at 
grade 4); the other grades increased, but only slightly. 

e Between 2012-13 and 2014-15 in reading, grade 4 improved but grades 3 and 5 did not. 
Grade 3 and 4 students made gains in 2013-14 only to fall back in 2014-15. 


Wilburn had higher proficiency percentages in reading than the other Renaissance schools before 
the program began. Between 2007-08 (before TAP) and 2011-12, the other Renaissance schools 
showed increases in proficiency in reading which were larger than those seen at Wilburn. From 
2012-13 to 2014-15, increases in proficiency occurred only at grade 4 for both Wilburn and the 
comparison schools; the grade 4 increase at Wilburn was slightly larger than at the other 


20 


TIF at Wilburn Year 5 DRA Report No. 15.12 


Renaissance schools. Overall, there was little difference in the composite scores for Wilburn 
versus the other schools. Results suggest TIF/TAP did not result in demonstrably greater 
improvements in reading proficiency over time compared to similar schools without the 
program. 


Table 13 
Reading: Percent of Students at or Above Grade Level at 
Wilburn and Other Renaissance Schools 


2013-14 
TIF Yr 4 


2014-15 
TIF Yr5 


38.4 46.9 oe 


Wilburn 2907-08 | 2008-09 2009-10 2010-11 2011-12 2012-13 
Grade Pre TAP Yri TAPYr2_ TIF Yr1* TIF Yr 2 TIF Yr3 


40.7 
60.0 
61.9 


2008-09 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15 


46.3 
47.4 
47.4 


42.8 
48.4 
48.7 


44.2 40.5 46.5 


Composite 46.6 
Note 1: * 2010-11 was the third year of TAP, 2011-12 was the fourth year, and so on. 

Note 2: Standards increased and assessments were changed in 2012-13. Comparisons cannot be made with validity 
between 2011-12 and 2012-13. 

Note 3: By grade (rows), green shading shows increases from 2007-08 to 2011-12 and 2012-13 to 2014-15 of three 
percentage points or more. Base year column headings are highlighted in blue. 

Source: North Carolina EOG Assessment Level Scores (Level 3 or 4 - 2007-08 - 2011-12; Level 3, 4, or 5 - 2012-13- 
2014-15). 
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Figure 3 
Composite Reading EOG: Percentage of Students Proficient 
100.0 
$0.0 
80.0 
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@ 
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em@e= \WVilburn 53.0 60.5 54.6 55.6 56.9 40.2 43.2 40.4 
==@— Other Renaissance 38.2 47.1 46.6 54.0 58.0 40.3 39.7 40.9 
Year 
Math 
The math target for 2011-12 was 75%; this target was met at grades 4 and 5 
(but not grade 3). The target increased to 90% for 2014-15; this target was EOG data suggest 
not met and was not realistic with the revised curriculum and assessments. In that TAP did not 
terms of change over time in math, Table 14 shows increases in half (three of result in 
six) grade level comparisons over the two time spans: demonstrably 
greater 


e Between 2007-08 and 2011-12, third grade proficiency declined, 
while fourth and fifth grade increased slightly; grade 5 increased the 
most. 

e Between 2012-13 and 2014-15, grade 4 saw increases in proficiency, 
grade 5 saw little change, and grade 3 saw a large decline. 


improvements in 

reading or math 

proficiency over 
time. 


Compared to other Renaissance schools, Wilburn started with higher proficiency percentages but 
fell to the math achievement level of the other schools over time. Wilburn had fewer positive 
increases (of three percentage points or more). Again, this suggests TAP did not result in 
demonstrably greater improvements in math proficiency over time than similar schools without 


TAP. 
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Table 14 
Mathematics: Percent of Students At or Above Grade Level 
Wilburn and Other Renaissance Schools 


2007-08 | 2008-09 2009-10 2010-11 2011-12 | 2012-13 2013-14 2014-15 


Wilburn 
Grade Pre “ ” se we Les TIFYr2 | TIFYr3 TIFYr4 TIF Yr 
3 75.7 5130 711 52.4 54.9 327 
4 77.9 69.2 624 316 36.5 as 
3 68.5 62.7 68.1 43.7 39.4 42.4 
Composite 735 | 77.1 613 673 760 | 418 441 411 


Other Renaissance Schools (program began in 2011-12) 
Grade 2007-08 | 2008-09 2009-10 | 2010-11 2011-12 | 2012-13 2013-14 2014-15 
3 49.4 65.2 59.0 68.0 47.9 


4 49.6 57.5 68.8 71.0 42.6 
5 55.4 41.5 


Composite 51.3 


Note 1: Standards increased and assessments were changed in 2012-13. Comparisons cannot be made with validity 
between 2011-12 and 2012-13. 

Note 2: By grade (rows), green represents an increase from 2007-08 to 2011-12 and 2012-13 to 2014-15 of three 
percentage points or more. 

Source: North Carolina EOG Assessment Level Scores (Level 3 or 4 2007-08 - 2011-12, Level 3, 4, or 5 2012-13- 
2014-15). 


Figure 5 
Composite Math EOG: Percentage of Students Proficient 
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Growth 


The best way to examine improvement from 2011-12 through 2014-15 is through EVAAS 
growth results, since changes in the performance category standards do not affect the analysis 
results. TAP/TIF set goals for growth based on EVAAS for grades 4 and 5. 


EVAAS results by grade and subject are shown in Table 15. For 2014-15, Wilburn met state 
expectations in reading and math at grade 5 but failed to do so in reading or math at grade 4. In 
terms of student growth (where a student is compared to himself or herself over time with results 
aggregated to the school level), Wilburn's EVAAS data for 2014-15 revealed that: 


e Fourth graders had strong growth in 2012 in both reading and math, exceeding 
expectations, but growth ratings declined over time. 

e Fifth graders met growth expectations in reading every year, and exceeded them in math 
until 2015, when they met expectations. The pattern of yearly results at Wilburn was 
most positive in grade 5 math. 

e Compared to the other Renaissance schools, Wilburn stood out as more positive only for 
5" grade math. 


Overall, Wilburn results were at or above state expectations each year except for grade 4 in both 


subjects in 2015, but not consistently stronger with TAP compared to similar schools without 
TAP. 
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Table 15 
EVAAS Results Year to Year 2012 to 2015 


Reading — Grade 4 $2012 $2013 $2014 $2015 
Barwell Road 
Brentwood 
Creech Road 
Wilburn 


Math — Grade 4 $2012 $2013 $2014 $2015 
Barwell Road 
Brentwood 
Creech Road 
Wilburn 


Reading — Grade 5 $2012 $2013 $2014 $2015 
Barwell Road 
Brentwood 
Creech Road 
Wilburn 


Math — Grade 5 $2012 $2013 $2014 $2015 
Barwell Road 
Brentwood 
Creech Road 
Wilburn 


Exceeds expected growth; progress significantly above the average school 


Blue ‘ 
in the state. 
Green Meets expected growth. 
Red Does not meet expected growth; progress significantly below the average 


school in NC. 
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A target set by the school was that 90% of fourth and fifth graders would show at least one year 
of growth in reading and math based on SAS® EVAAS scores of 2.5 or above. 


As shown in Table 16, these lofty targets were not met in either reading or math. The percentage 
of students showing one year of growth was highest for math at grade 5. Results fluctuated year 
to year more in reading than in math with the new EOG tests in place (from 2012-13 through 
2014-15). 


Table 16 
Percentage of Students Earning At Least One Year of Growth on NCE Scores 


2010-11 2011-12 
Grade TIF Yr 1* TIF Yr2 


2012-13 2013-14 2014-15 
TIF Yr 3 TIF Yr 4 TIF Yr5 


. 4 NA 53% 42% 55% 40% 
Reading 

5 35% 62% 48% 61% 52% 

Math 4 NA 79% 49% 43% 45% 

43% 74% 60% 60% 63% 


Note: Standards increased and assessments were changed in 2012-13. Comparisons cannot be made 
with validity between 2011-12 and 2012-13. 
Source: North Carolina EOG Assessment NCE scores 2008-09 to 2014-15 


As shown in Table 17, only one of the seven WCPSS targets set for TIF was partially met in year 
5 of the grant. 


Annual goal targets were set for the TIF grant and reported to the funder as part of the Annual 
Performance report. Targets for year 5, as shown in Table 17, show that only one of the seven 
targets was partially met (classroom growth at grade 5). Actual performance came close to the 
target for several others. Targets for EOG proficiency were unrealistic given new EOG 
assessments with higher standards starting in 2012-13. Goal 6 was lofty. Thus, results for targets 
were not very positive. 
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Table 17 
TIF Targets and Outcomes for Year 5 (2014-15) 


Target Focus Target Actual 


1. A. Teachers earning a performance award 


959 39 
B. The percentage of principals in high-need schools who have a ‘i aa 
. 100% 0% 
record of effectiveness. 
2. Teachers remaining at Wilburn Elementary during the following 
4s 95% 92% 
year (stability vs. turnover) for competent teachers 
Gr 475% 
3. Grade 4-5 teachers— t showi | level th 1009 
rade eachers— percent showing classroom level grow % Gr 5 100% 
4. Students in grades 3-5 scoring at or above grade level in 85% reading 40% reading* 
reading/math as measured by End of Grade Assessments 90% math 41% math* 
5. Students in grades K-5 making a 5-point increase in their 80% reading 71% reading 
reading and math MAP RIT scores based on fall to spring testing 80% math 77% math 
. fo} 
6. Students in grades 4 and 5 earning at least one year of growth é . pg ae 
d by SAS® EVAAS NCE scores of 2.5 or greater in spe ligl oe 
nen u ere 90% math Gr.4 45% 
ue Gr.5 63% 
7. Teachers receiving evaluations of proficient or exemplary based 95% 93% 


on the TAP instructional rubric 
Note 1: Targets met are shaded and highlighted in a green font in the “Actual” column. 
Note 2*: EOG proficiency standards for on grade level performance became more rigorous with new 
assessments in 2012-13, greatly increasing the difficulty of reaching the targets. 


Conclusions 


Based on survey results, interviews, meetings, and observations, Wilburn implemented TAP and 
the four principles upon which it is based with reasonable fidelity across the five years TIF was 
in place. Issues related to timing of evaluations and feedback sessions improved over time, 
although they were not completely resolved. Turnover in principals, Mentor/Master teachers, 
and regular teachers impacted the continuity and implementation of TAP because of re-training 
needed for new staff as well as building trust and strong working relationships among 
colleagues. Teachers had mixed opinions on the value of TAP. Some perceptions became 
significantly more positive over time (such as the value of the TAP instructional rubric, the 
value of the Mentor and Master teachers, and TAP helping teachers identify student needs and 
how to address them), but most did not change significantly over time. Approximately three- 
quarters of the teachers did not view TAP as helpful overall. Comments that TAP was too much 
work, was causing teachers to leave, was too much of a stressor, and that evaluations were not 
fair continued to be made by some teachers each year. 


Teacher and principal compensation systems were revised to provide performance pay for 
improved teacher effectiveness ratings and growth in achievement. A high percentage of teachers 
received a bonus for at least one component each year after year 1. The goal was that 95% of 
teachers would meet the standard set for observation criteria; Wilburn fell just short at 93% (49 
of 53). In focus groups, many teachers commented that performance pay did not influence their 
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decision to work at Wilburn. Additionally, despite high percentages of teachers receiving 
bonuses each year, teacher turnover remained above the goal of 5% all but one year, with the 
highest rates when Renaissance was added to the school. 


EOG reading and math results were not more positive consistently compared to similar schools 
without TAP/TIF in terms of student proficiency and growth. Grade 5 math growth was the one 
case that showed a positive edge for Wilburn. K-2 literacy results were stronger than the 
comparison schools in 2014-15, but not in 2013-14. Thus, Wilburn results with TAP did not 
stand out as consistently stronger than the other schools without TAP (but with Renaissance 
resources) at grades 3-5. 


Overall, despite implementation of all principles of TAP, TAP/TIF results were somewhat 
positive for teachers’ instructional quality ratings and pay, but not for student achievement in 
most cases. Nearly all grant targets were not met for year 5. While other factors contributed, a 
majority of teachers saw TAP as a burden in terms of workload, extra testing, and extra 
observations, with very little benefit gained from these activities. Thus, overall, TAP/TIF did not 
show positive results in terms of the costs and benefits. The fact that the grant and model were 
in only one school limits our ability to generalize the results found beyond Wilburn. 


Next Steps/Recommendations 


TIF grant funding extension is ending in 2015-16 and Wilburn is now one of the Elementary 
Support Model schools. Wilburn has maintained positive aspects of TAP—such as the 
instructional rubric, lesson modeling, and coaching through the provision of grade level coaches. 
The professional learning model has been revised. The principal is also working to develop 
greater trust and collaboration among teachers, coaches, and the administration, which she feels 
were negatively impacted by TAP. 


Moving forward, we recommend that WCPSS staff: 


e Are careful in selecting grants to apply for and which schools will participate. Adding 
TAP/TIF and then Renaissance at Wilburn created a heavy load for Wilburn staff, 
especially since most WCPSS requirements continued. The cost/benefit in terms of 
human capital as well as actual funds must be considered; if a grant will place an 
unreasonable burden on the schools, the grant might not be beneficial. In addition, 
having a grant that applies to only one school can limit the amount of expertise and 
support available from staff in Central Services and other schools. Finally, targets must 
be realistic yet challenging; adjusted targets should be negotiated if major changes in 
curriculum and assessments take place. 


e Realize performance pay has had mixed effects in large research studies and is difficult to 
implement fairly (Townsend, Baenen, & Germuth, 2015). The value of performance pay 
was not supported by teacher turnover or achievement results at Wilburn. Teachers 
reported performance pay was not an important factor in their decision to work at 
Wilburn. Whether this is related to the small amount of money at stake per teacher, the 
extra work involved, or other factors is unknown. 


28 


TIF at Wilburn Year 5 DRA Report No. 15.12 


References 


Agam, K., Reifsneider, D., & Wardell, D. (2006). The Teacher Advancement Program (TAP): 
National Teacher Attitudes. Washington, DC: Teacher Advancement Program 
Foundation. 


Good, R. H., & Kaminski, R. A. (Eds.). (2002). Dynamic Indicators of Basic Early Literacy 
Skills (6th ed.). Eugene, OR: Institute for the Development of Educational Achievement. 
Available: http://dibels.uoregon.edu/ 


Greene, J. C., Caracelli, V. J. and Graham, W. F. 1989. Toward a conceptual framework for 
mixed-method evaluation design. Educational Evaluation and Policy Analysis, 11(3), pp. 
255-74. 


Northwest Evaluation Association (NWEA). (2009, October). Technical manual for 
Measures of Academic Progress and Measures of Academic Progress for primary 


grades. Lake Oswego, OR: Author. 


Townsend, M. (2013). Renaissance schools evaluation, year 1 (2011-12). Raleigh, NC: Wake 
County Public School System. 


Townsend, M., Baenen, N., & Germuth, A. (2015). Project Renaissance final evaluation, 
2011-2012 to 2013-2014. Raleigh, NC: Wake County Public School System. 


29 


TIF at Wilburn Year 5 DRA Report No. 15.12 


Appendix A: TIF Evaluation Plan Year 5 
Appendix B: TIF Teacher Survey 


Appendix C: TIF Teacher Focus Group Questions 


30 


TIF at Wilburn Year 5 DRA Report No. 15.12 


Appendix A 
TIF Evaluation Plan Year 5 


Evaluation Plan: WCPSS TIF Grant 
Year 5 (2014-2015) 


The WCPSS TIF Grant is designed to a) increase teacher effectiveness; b) increase student 
learning and accelerate their learning growth; and c) close the achievement gap between students 
at Wilburn and WCPSS. Evaluation of the WCPSS TIF Grant will be both formative and 
summative in nature. The formative evaluation began at the start of the grant and will continue 
throughout. The summative evaluation began formally at the end of Year 1 and will continue as 
well through the life of the grant. Formative evaluative activities will be undertaken to broadly 
assess how well the grant is unfolding and potential drivers of or barriers to the grant’s success. 
Summative evaluation activities will address to what degree expected outcomes have occurred. 
In some cases data collected formatively will be used in the summative evaluation and vice 
versa. 


Evaluation Questions 
Formative evaluation questions to be addressed in this evaluation include: 
1. What were the changes in staff composition at Wilburn for 2014-15? 
2. What is the fidelity of implementation of TAP at Wilburn Elementary School? 


3. Are there drivers of or barriers to the fidelity of implementation of TAP, and if so, what 
are they? How are barriers being addressed? 


4. To what degree do teachers view TAP as a supportive intervention? What aspects of TAP 
do they like best / see as most useful or like least / see as least useful? 


5. To what degree do teachers report that TAP is improving their content knowledge and 
practice / pedagogy? 


6. What changes would teachers like to see in how TAP is implemented? 


7. What assessments are teachers using for formative purposes? Which do they find most 
helpful and are any more work than they are worth? 


8. How do students view TAP and their experiences at Wilburn? 


9. What are differences in teachers’ perceptions by level (K-2, 3-5) or year’s involvement 
with TAP? 
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Summative evaluation questions were derived primarily from the project performance measures 
and targets as listed in the 2014-15 Annual Performance Report (APR / Ed 524b). The specific 
targets are listed in the next section--Evaluation Activities and Timeline. In addition, the 
evaluation examined these summative questions: 


1. What percentage of teachers earn a bonus overall? How do these results compare to other 
TAP schools in North Carolina? How does this compare to other Renaissance schools in 
WCPSS? 


2. What perceptions do students at Wilbur have of the changes made at the school? What 
progress have they made over time? 


3. What are trends in terms of TIF impacts and what aspects of the TIF grant/TAP are worth 
continuing after the project ends? 


Evaluation Activities and Timeline 


Evaluation activities will include observation of the TAP program at Wilburn, review and 
analysis of test scores (MAP, EOG, etc.) and EVAAS growth data, formative and summative 
surveys of teachers (self-developed), focus groups/interviews with students, and teacher and 
administrator focus groups and interviews. Additionally, this year, the evaluator will observe 
teachers at a comparative school in Wake County. 


Observations will be ongoing throughout the school year and include observations of TAP 
meetings led by administrators, Master, and Mentor teachers, and Master and Mentor teachers 
working with and coaching Wilburn staff. A survey will be conducted in May 2015 to gather 
quantitative ratings of how teachers felt about TAP overall for the 2014-15 school year. These 
findings will be compared to May 2014-15 findings. Teacher and administrator focus groups will 
be conducted in January 2015 to further understand staff experiences with and feelings about 
TAP. MAP data will be reviewed as are available to determine areas in which corrections are 
needed to better ensure students’ growth; EOG scores and growth data will also be reviewed to 
better understand TAP’s impact on student achievement. 
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Below is a cross walk that identifies the evaluative activities associated with each evaluation 
question and the timeline in which the evaluation activities will occur. Specific summative 
targets are also shown. Some increased across the years of the grant. These reflect the targets 
for 2014-15 specifically. 


Siena Evaluative Activity Timeline 
What is the fidelity of Observations of TAP e Ongoing 
implementation of TAP at training, meetings, and e As implemented 
Wilburn Elementary School? coaching e May 2015 

TAP Survey e January 2015 
Evaluator-developed 
survey 
Teacher focus groups 
Are there drivers of or barriers Observations of TAP e Ongoing 
to the fidelity of training, meetings, and e As implemented 
implementation of TAP, and if coaching e January 2015 
so, what are they? How are Evaluator-developed 
barriers being addressed? survey 
Teacher focus groups 
To what degree do teachers Observations of TAP e Ongoing 
view TAP as a supportive training, meetings, and e May 2015 
intervention? What aspects of coaching e January 2015 
TAP do they like best / see as Evaluator-developed 
most useful or like least / see survey 
as least useful? Teacher focus groups 
To what degree do teachers Observations of TAP e Ongoing 
report that TAP is improving training, meetings, and e As implemented 
their content knowledge and coaching e May 2015 
practice / pedagogy? TAP Survey e January 2015 
Evaluator-developed 
survey 
Teacher focus groups 
What changes would teachers Observations of TAP e Ongoing 
like to see in how TAP is training, meetings, and e As implemented 
implemented? coaching e May 2015 
TAP Survey e January 2015 
Evaluator-developed 
survey 
Teacher focus groups 
What are differences in Observations of TAP e Ongoing 
teachers’ perceptions by level training, meetings, and e As implemented 
(K-2, 3-5) or year’s coaching e May 2015 
involvement with TAP? TAP Survey e January 2015 
Evaluator-developed 
survey 
Teacher focus groups 
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Summative Evaluation Questions 


Evaluative Activity 


Timeline 


1. To what degree do 95% of 
teachers earn a performance 


award based on their score on the 


Review of TAP 
evaluation ratings and 
teacher designations 


June 30, 2015 or as 
available 


TAP rubric of the grant and remain for 2013-14 
teaching at Wilburn Year- Round 
Elementary School during the 
following year? 
2. Towhat degree do 100% of Grade | e Review of EVAAS e June 30, 2015 or as 
5 teachers make classroom level scores available 


growth (defined as an EVAAS 
score of 3 or 4) and thus earn a 


growth-based performance award? 


3. To what degree do 85% of 
students in grades 3-5 score at or 
above grade level in reading as 
measured by End of Grade 
Assessments? 

4. Towhat degree do 90% of 
students in grades 3-5 score at or 
above grade level in math as 
measured by End of Grade 
Assessments? 


Review of EOG scores 


Review of EOG scores 


June 30, 2015 or as 
available 


June 30, 2015 or as 
available 


5. To what degree do 80% of 
students in grades K-5 make a 5- 
point increase in their reading 
MAP RIT score based on fall to 
spring testing? 


Review of MAP scores 
fall and spring 


June 30, 2015 or as 
available 


6. To what degree do 80% of 
students in grades K-5 make a 5- 
point increase in their math MAP 
RIT score based on fall-spring 
testing? 


Review of MAP scores 
fall and spring 


June 30, 2015 or as 
available 


7. To what degree do 90% of e Review of EVAAS e June 30, 2015 or as 
students in grade 5 earn at least scores available 
one year of growth as measured 
by SAS EVAAS scores of 2.5 or 
greater in math? 
8. To what degree do 90% of e Review of EVAAS e June 30, 2015 or as 
students in grade 5 earn at least scores available 
one year of growth as measured 
by SAS EVAAS scores of 2.5 or 
greater in reading? 
9. To what degree do 95% of e Review of TAP e June 30, 20145 or as 


teachers receive evaluations of 
proficient or exemplary based on 
the TAP instructional rubric? 


evaluation ratings 


available 


10. How did the composition of 
Wilburn's student body change in 
14-15? How did the staff and 


teacher makeup change? At which 


grades were the prior Wilburn 
teachers kept? 


Review of teacher 
designations for 2013- 
14 


June 30, 2015 or as 
available 
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11. Did students feel their teachers e Studentinterviews and |e Ongoing 
utilized teaching / learning review of academic 
strategies that helped them data 


succeed? Did they feel engaged in 
their learning? Did they think TAP 
features made a positive 
difference to their learning? 

12. How did the results for Wilburn in e Review of EOG scores |e June 30, 2015 or as 
2014-15 compare to past years? available 


Evaluation Methodology 


Both types of evaluation, formative and summative, will utilize a mixed-methods approach 
whereby qualitative and quantitative methods are mixed or inter-connected, where possible, to 
improve validity and reliability of results. Among the purposes for mixed-method evaluation 
design, Green et al. (1989) highlight five major ones, four of which are applicable to the 
evaluation of the TAP program that might enhance the evaluation as follows: 


Triangulation tests the consistency of findings obtained through different instruments. 
Triangulation will increase chances to control, or at least assess, some of the threats or 
multiple causes influencing our results. 


Complementarity clarifies and illustrates results from one method with the use of another 
method. In our case, observation will add information about the TAP process and will 
qualify test findings. 


Development results from one method shape subsequent methods or steps in the research 
process. In our case, partial results from the preprogram measures might suggest that 
other assessments should be incorporated. 


Initiation stimulates new research questions or challenges results obtained through one 
method. In our case, in-depth interviews with teachers and principals will provide new 
insights on how the program has been perceived and valued across sites. 


Reporting 


The evaluator will work to keep stakeholders aware of results as data are collected and analyzed. 
Thus throughout the evaluation the evaluator will report findings on a periodic basis, especially 
as is needed for stakeholders to make corrective changes. All evaluation activities and findings 
will be summarized as part of an annual final report, where the context of the evaluation will also 
be described. Evaluation findings will be compared over time in future reports to better 
understand the summative changes that are or are not occurring. A summary for local use will 
highlight key results over time. 
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Appendix B 
TIF Teacher Survey 


Teacher Incentive Fund (TIF) Survey 


Wilburn Elementary School 
May 2014 


This survey is designed to assess how TAP is unfolding in Wilburn this year and the degree to 
which you believe it is effecting positive change in you, your students, and your school. Please 
take 10 minutes to respond to the questions below. Responses will be kept confidential meaning 
that data will be reported in aggregate form only with all names and identifying characteristics 
removed. If you have questions about this survey and/or its use, please feel free to contact Amy 
Germuth at (919) 401-5403 or AmyGermuth@EvalWorks.com or Nancy Baenen at Wake 
County Public Schools at nbaenenl@wcpss.net. 


1. Please select the response that best describes the degree to which and where you use 
TAP instructional methods (e.g., methods learned from professional development 
provided by Master teachers): 


lxa 
Daily ake eaee month Never 
week month 
or less 
a. To what degree do you utilize TAP instructional 
aa y O O O O O 
methods in your lessons? 
All 3-4 2 1 No 
subjects | subjects | subjects | subject | subjects 
b. To what degree do you utilize TAP instructional 
eae O O O O O 
methods across your subjects? 


2. Please indicate your agreement on a scale of 1 (Strongly Disagree) to 5 (Strongly Agree) 
to the following statements about TAP professional development (e.g., the professional 
development provided by Master teachers). 


1 2 3 4 5 
TAP professional development Strongly Diguiee | enrery | anne Strongly 
Disagree Agree 
a. is aligned to our school's needs. O O O O O 
b. is aligned to Common Core Standards. O O O O O 
c. helps me engage my students better than before. O O O O O 
d. ae provided me with techniques to help me O O O O O 
etter group students. 
e. has made me better able to motivate my O O O O O 
students. 
f. has improved my teaching skills. O O O O O 
g. has made me a more effective teacher. O O O O O 
h. has allowed me to better identify students O O O O O 
needing extra support. 
i. has allowed me to better identify the type of O O O O O 
support my students need. 
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is meeting my needs as a teacher. 

k. has made me more knowledgeable about my 
students. 

1. has improved my ability to help my students to 
use higher order thinking skills. 

m. has improved my ability to help my students to 
use better problem-solving skills. 

n. will help student achievement increase at 

Wilburn this year. 


=n 


CO;}O;O] 040 
CO;}O;O] 0 4j0 
CO;}O;O] 0 jo 
O;}O;O0] 040 
CO;}O;O] 040 


3. Please indicate your agreement on a scale of 1 (Strongly Disagree) to 5 (Strongly Agree) 
to the following statements. 


1 D 3 4 5 
S I : S I 
pee Disagree | Neutral Agree ee 
a. Iam highly knowledgeable of the TAP O O O O O 
instructional rubric. 
b. Iregularly use data to track my students' O O O O O 
progress. 
c. Master teachers are available to support me if I O O O O O 
need help. 
d. Mentor teachers are available to support me if I O O O O O 
need help. 
e. Master teachers have the experience necessary O O O O O 
to help me improve my teaching skills. 
f. Mentor teachers have the experience necessary O O O O O 


to help me improve my teaching skills. 

g. The professional development designed and 
provided by Master teachers helps me improve O O O O O 
my instruction. 

h. The professional development designed and 
provided by Mentor teachers helps me improve O O O O O 
my instruction. 

i. The administration at my school is supportive 


of TAP. © co 0 
j. The administration at my school is supportive O O O O O 
of me. 
k. TAP is a positive aspect of being at Wilburn. O O O O O 


4. Please indicate your agreement on a scale of 1 (Strongly Disagree) to 5 (Strongly Agree) 
to the following statements. 


1 2 3 4 5 
Strongly ‘ Strongly 
Disavae Disagree | Neutral Agree pee 
a. I feel comfortable being evaluated using the 
ng 8 O O O O O 
TAP rubric as the basis for the evaluation. 
b. Evaluations using the TAP rubric are bein 
ee 8 O O O O O 
conducted in a fair manner. 
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c. Post-evaluation conferences are being 


eae O O O O O 

conducted in a timely manner. 

d. Ibelieve that the Master Teachers have the 
necessary training to conduct evaluations using O O O O O 
the TAP rubric. 

e. I believe that the Administrators at Wilburn 
have the necessary training to conduct O O O O O 
evaluations using the TAP rubric. 

f. I believe that the Mentor Teachers have the 
necessary training to conduct evaluations using O O O O O 
the TAP rubric. 

g. Iagree with the TAP ratings I have received O O O O O 


for my evaluations. 

h. Iwas able to adequately learn the TAP 
instructional rubric before TAP evaluations O O O O O 
were conducted. 

i. TAP evaluation feedback is meaningful to me 


O O O O O 
as a teacher. 
j. I feel like I have a say in the TAP evaluation O O O O O 
process. 
k. The TAP evaluation process has helped me O O O O O 


grow professionally. 


5. To what degree have the following assessments been helpful in determining your 
students’ needs? If you do not use an assessment please indicate DK for Don't Know. 


Don't t 2 d 
Karak Not ee at aL Very helpful 
a. DPI Math Assessments (Grades K-2) O O O O 
b. mCLASS (Grades K-2) O O O O 
c. DIBELS O O O O 
d. MAP O O O O 
e. CASE 21 O O O O 


6. To what degree have the following assessments been helpful in monitoring student 
rogress? If you do not use an assessment please indicate DK for Don't Know. 


Don' 1 2 3 
aed Not ae at Seeds Very helpful 
a. DPI Math Assessments (Grades K-2) O O O O 
b. mCLASS (Grades K-2) O O O O 
c. DIBELS O O O O 
d. MAP O O O O 
e. CASE 21 O O O O 


7. Are you new or returning teacher at Wilburn this year? 
O I am a new teacher at Wilburn this year 
O 1am a returning teacher to Wilburn this year 
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8. For new teachers only: 


Please indicate your agreement on a scale of 1 (Strongly Disagree) to 5 (Strongly Agree) to the 
following statements: 


1 2 3 4 5 
Strongly ; Strongly 
Disagree Disagree | Neutral Agree gree 
a. Staff morale is high. O O O O O 
b. Students demonstrate good behavior. O O O O O 
c. Student achievement is improving. O O O O O 


9. For returning teachers only: 
Please indicate your agreement on a scale of 1 (Strongly Disagree) to 5 (Strongly Agree) to the 
following statements: 


1 2 3 4 5 
Strongly ; Strongly 
Disagree Disagree | Neutral Agree Agree 
a. Staff morale has improved this year. O O O O O 
b. Student behavior has improved this year. O O O O O 
c. Student achievement is improving this year. O O O O O 


13. Please describe how TAP has impacted your professional practice: 


14. Please describe how you feel TAP has impacted Wilburn: 


15. Is TAP better, the same, or worse this year than in the past? 


16. Please indicate your role at Wilburn: 

O Teacher 

O Teacher/Mentor Teacher 

O Master Teacher 

O Other education professional (e.g., counselor, media specialist, etc.) 


17. Please indicate the grade(s) that you work with (feel free to select as many that apply): 
O Kindergarten - Grade 2 
O Grade 3-5 


18. Please indicate your name. This is for monitoring return rate purposes only. Only the 


evaluator, Amy Germuth, will see your name linked to your actual responses. Names will be 
deleted from the file for analysis purposes. 
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Appendix C 
TIF Teacher Focus Group Questions 


TIF Teacher Focus Group Questions 
December 2013 


8. 


9. 


What has been the value of the TAP program to you personally? 


. What has been the value of the TAP program to Wilburn staff? 


What has been the value of TAP for Wilburn students? 
What have been the drawbacks of the TAP program to Wilburn? 


Knowing what you know now, how would you change how TAP was implemented at 
Wilburn? 


Overall, was TAP positive for Wilburn? Why or why not? 


As the TIF funding ends, there will be no funding available to continue TAP. What aspects of 
TAP do you think should be retained? 


Do you think the job-embedded training was a good model? 


What professional development do teachers at Wilburn need that TAP hasn’t addressed? 


10. Is there anything else that you would like to add? 
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